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The present invention relates to logic circuit emulation 
systems. In particular, the present invention relates to 
providing data transport across practically asynchronous 
portions of a logic circuit emulation system. 



2 . Discussion of the Related Art 

A typical emulation system for a large logic circuit is 
described, for example, in U.S. Patent 5,761,484, entitled 
"Virtual Interconnections For Reconf igurable Logic Systems," to 
Agarwal et al . Such an emulation system is often used during 
the development of an integrated circuit to simulate circuit 
operation and circuit performance. In such a system, the 
designer provides a logic netlist that is then partitioned by 
the emulation system for implementing an emulation circuit 
configured in a number of programmable logic devices (e.g., 
field programmable gate arrays or FPGAs) . These programmable 
logic circuits (PLDs) are typically provided on one or more 
circuit boards in the emulation system, with each circuit board 
containing a number of these programmable logic devices 
connected in a convenient topology. 

Many techniques for efficiently implementing the emulation 
cirruit ha ve been deve loped. For example, U.S. Patent 
5, 761, 484, entitled "Virtual Interconnections ifmr-^R^x^^igurable 
Logic Systems" to Agarwal et al . , provides an efficient method 
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to route signals between the PLDs by "multiplexed data 
transport, " i.e., sharing input or output pins among many input 
or\output signals. In one implementation of that system, a 
clocks^signal ("virtual clock") of many times the frequency of 
the sysr^m clock is used for these input and output signals. 
U.S. PatentN5, 854, 752, entitled "Circuit Partitioning Technique 
For Use With Mta^tiplexed Interconnections" to Agarwal, provides 
an efficient way oi circuit partitioning that achieves high 
utilization of the available resources in the PLDs. U.S. 
Patents 5, 659, 716 and 5,8*50^ 537, both entitled "Pipelined Static 
Router And Scheduler For Configurable Logic System Performing 
Simultaneous Communications and CotrasQunication'' to Selvidge et 
al . , disclose methods for ef f iciently rb^ting among PLDs signals 
under timing constraints. U.S. Patent 5.82 0>348, entitled 
"Logic Analysis System For Logic Emulation Syste?ft§" to Stewart 
et al . , provides logic analyzer functions to be usecTvn 
analyzing the operations within the emulation circuit. 



In a large logic circuit, circuit operations are controlled 
by oik or more clock signals. Thus, proper handling of clock 
2J? signalsNLs important to achieve a successful emulation of a 
: ;^f logic circ^sd-t. For example, U.S. Patent 5,649,176, entitled 
f~. "Transition Analysis And Circuit Resynthesis Method and Device 
□ For Digital Circuit Modeling," discloses using an internal clock 

signal outside of\the timing signals of the logic circuit to 
25 control the internal\pperations of the emulation circuit. In a 
typical emulation system, a single clock signal is distributed 
^"throughout the emulated l>sgic circuit to provide 

synchronization. While this\clock distribution scheme is 
conventional in an emulation crg^cuit configured in PLDs in very 
30* close proximity (e.g., PLDs on a single circuit board, or on 
different circuit boards interconnected on a single backplane 
bus) , such a clock signal cannot be provided between PLDs 
separated by a relatively large distance ^e.g., PLDs on circuit 
boards on different chassises) or at high clbck frequencies, 
3 5 such as those used for multiplexed data transput. In such a 
system, there may be large clock skews at different points of 
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the system relative to the clock period that cannot be reliably 
estimated. Thus, practically, those different points of the 
^s^eoi.are effectively asynchronous relative to each other. 
Thus, there is^a need— iox, .^jreliable method for transporting 
5 Q^data between distinct synchronous"~c?oT?lp^ a system, 

without relying on a common clock signal distribu^^*--^i^ughout 
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Asynchronous communication can be carried out by: (a) 
providing explicit flow control signals, (b) embedding a clock 
signal in a data signal, and extracting the clock signal in a 
decoding circuit during decoding, and (c) providing a frequency- 
controlled clock signal, and encoding both data and clock phase, 
and reconstructing clock signal phase during decoding. 



SUMMARY OF THE INVENTION 

jij The present invention provides methods and systems for 

Lj reliably transmitting data across two emulation systems that are 

;! = substantially asynchronous relative to each other. 

i'ff According to one embodiment of the present invention, 

Q method for transmitting a data packet between asynchronous 
2$ systems includes: (a) providing a transmit clock signal of a 
predetermined frequency; (b) transmitting a framing sequence 
serially over a connection between the asynchronous systems, in 
accordance with the transmit clock signal; and (c) subsequent to 
transmitting the framing sequence, transmitting the data packet 
25 serially over the connection. Under that method, each bit in 

the framing sequence and the data packet is transmitted over two 
transmit clock periods. Symmetrically, one embodiment of the 
present invention provides a method for receiving a data packet 
between asynchronous systems, which includes: (a) providing a 
30 receive clock signal of a predetermined frequency; (b) detecting 
a framing sequence transmitted serially over a connection 
between the asynchronous systems, in accordance with a receive 
clock signal; and (c) subsequent to receiving the framing 
sequence, receiving the data packet serially over the 
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connection. Under that receiving method also, each bit in said 
framing sequence and said data packet is received over two 
receive clock periods . 

According to another aspect of the present invention, an 
5 emulation system is provided that includes: (a) a circuit board 
provided with programmable logic devices for implementing an 
emulation circuit and a transceiver circuit, the. circuit board 
receiving a clock signal of a predetermined frequency; (b) a 
controller coupled to a host computer, the controller having a 
10 transceiver circuit for communicating with the transceiver 

circuit of the circuit board and also receiving a clock signal 
of the predetermined frequency; and (c) a connection between 
£3 said transmitter circuit and the receiver circuit. In this 
;=3 emulation system, each bit of data transmitted over the 
3L$ connection has a duration of two or more periods of the clock 
hJ signal received at the circuit board. In one implementation, 
!Is the clock signal received at the circuit board and the clock 
LJ signal received at the controller are provided by a common 
;;L source. Alternatively, the clock signals for the transmitter 
20 circuit and the receiver circuit are generated independently. 

Such a clock signal can be provided by a virtual clock signal, 
!=S or can be provided by a clock signal twice the frequency of the 
□ virtual clock signal. Using a transmit clock signal at twice 

the frequency of the virtual clock signal allows data to be 
25 transmitted at the virtual clock rate between the controller and 
the circuit board. 

In a second embodiment, the method of the present invention 
is applied to two circuit boards housed on different chassis of 
an emulation system. 

3 0 The present invention is better understood upon 

consideration of the detailed description .below and the 
accompanying drawings . 



BRIEF DESCRIPTION OF THE DRAWINGS 
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Figure 1 shows emulation system 100 in which multiplexed 
data transport methods of the present invention are applicable. 

Figure 2 shows transmit clock 2 01, data signal 202 and 
receive clocks 203, 204 and 205. 

Figure 3 shows a data packet transmitted over data signal 

202 . 

Figure 4a is a block diagram of transmitter circuit 400 
according to one embodiment of the present invention 

Figure 4b is state diagram 450 that illustrates the control 
operations of control circuit 405. 

Figure 5a is a block diagram of receiver circuit 500 in 
accordance with one embodiment of the present invention. 

Figure 5b is state diagram 550 showing the control 
operations of control circuit 506. 

Figure 6 shows circuit 600 that can be configured in an 
emulation circuit consisting of multiple circuit boards to 
effectuate data transfer. 

Figure 7 shows system 700 including emulation system 701, 
controller 702, and host system 750. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The present invention is applicable to an emulation system, 
such as that shown in Figure 1 . As shown in Figure 1, emulation 
system 100 includes two groups of circuit boards 101 and 102, 
each group having a number of circuit boards populated by field 
programmable gate arrays (FPGAs) which can be configured by 
controller 105 to emulate a user circuit. Signals between 
circuit board groups 101 and 102 are provided over a number of 
wires, such as wires 103 and 104 shown in Figure 1. Some of 
these signals can be signals in the emulation circuit configured 
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in circuit board groups 101 and 102 , and may be uni-directional 
or bi-directional. In this embodiment, circuit board groups 101 
and 102 are housed in different equipment chassis. Controller 
105 also controls the operation of circuit boards 101 and 102 
5 and receives selected signals from the emulation circuit 

configured in circuit board groups 101 and 102. Terminals 107 
and 108 represent, respectively, wires connecting logic signals 
from the emulation circuit configured in circuit board groups 
101 and 102 to controller 105. Controller 105 can communicate 
10 with host computer 106 over system bus 109, for example. 

According to one embodiment of the present invention, data 
can be communicated over terminals 103, 104, 107 and 108 without 

□ a common low- skew clock signal synchronized throughout emulation 
^ system 100. Instead, each of circuit board groups 101 and 102, 
%3 and controller 105 has access to a clock signal of a common 

! =3 predetermined frequency. Access to such a clock signal can be 

provided, for example, by transmitting a master clock throughout 
LJ the system, even though the phase relationship between any two 
points receiving this clock signal cannot be easily determined. 
20 In one embodiment, controller 105 receives a clock signal common 
j =y with one of circuit board groups 101 and 102. Alternatively, 
^ each device can generate a clock signal of the specified 

□ frequency locally. In one embodiment, each of circuit board 
groups 101 and 102 generates its own common frequency clock 

25 signal. In either situation, the phase of each clock signal in 
circuit board groups 101, 102 and controller 105 relative to 
each other is undetermined. For such clock signals, the total 
number of bits ("data size") per transmission is substantially 
given by the following constraint which is a function of the 

3 0 tolerance of frequency variation (Af ) : 




(2*data_size)*2**f*T<^-(T setup +T hold +T skew ) 
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where data_size is the number of bits in the transmission, 
T is the nominal clock period, T set up and T ho id are, respectively, 
the setup and the hold times, and T skew is the accumulated skew in 
the rise and fall times, due to propagation rate variations. In 
5 one embodiment, a data size of in excess of 100 bits is 

achievable. The data packet may be provided as fixed size or 
variable size. 

According to one embodiment of the present invention, data 
is sent between circuit board groups 101 and 102, and controller 
10 105 at one-half the predetermined frequency of the clock signal 
in these circuits. Figure 2 shows transmit clock 201, data 
signal 202 and potential receive clocks 203, 204 and 205. As 

□ shown in Figure 2, receive clock signals 203, 204 and 205 are 
;=2 respectively, 90° , 180° and 270° out of phase relative to 

transmit clock 201. In Figure 2, data signal 202 transitions at 
w the falling edges 211 and 212 of transmit clock signal 202, so 
?Z that each bit in data signal 2 02 remains valid for 2 cycles of 
i=J transmit clock 201. Note that, each of clock signals 203-205 
'!L has both a rising edge (e.g., edges 213, 215 and 218) and a 
20 falling edge (e.g., edges 214, 216 and 217) that is more than 

180° away from edges 211 and 212. By identifying an appropriate 
clock edge, data signal 2 02 can be sampled by any of receive 

□ clock signals 203, 2 04, 2 05 or any receive clock signal of an 
arbitrary phase relative to transmit clock 201. 

25 A phase recovery circuit 300 for a receiver detects a 

"framing sequence" transmitted on data signal 202. Figure 3 
shows the packet structure of data sent over data signal 2 02, in 
one embodiment of the present invention. During idle periods 
(i.e., when no data is transmitting), a logic "0" is transmitted 

30 on data signal 202. However, as shown in Figure 3, when a data 
packet is to be transmitted, framing sequence 301 is transmitted 
ahead of actual data 302. One or more parity bits 303 are sent 
to provide error detection. In one embodiment, the framing 
sequence is "01", so that each packet is separated by at least 

35 two receive clock cycles of logic w 0". 
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Figure 4a is a block diagram of transmitter circuit 400 
according to one embodiment of the present invention. As shown 
in Figure 4a, transmitter circuit 400 includes a data output 
circuit 401 which latches an n-bit data word from data bus 403 
5 according to clock signal 404. Output circuit 401 transmits the 
latched data according to a transmit clock signal (not shown) on 
serial line 4 07. In one embodiment, the transmit clock signal 
is half the frequency of clock signal 404, which is typically 
the virtual clock signal. Parity generation circuit 402 
10 computes one or more parity bits 4 06 to be transmitted with the 
output data on serial line 407. Control circuit 405 controls 
the operations of data output circuit 401 and parity generation 
circuit 402. 

Figure 4b shows state diagram 450 that illustrates the 
L5 control operations of control circuit 405. Initially, 
=3 transmitter circuit 400 is in an idle state 451 until "data 
jla ready" signal 408 is asserted to indicate valid data on data bus 
Ly 403. During this period, a logic "0" is repeatedly transmitted 

on serial line 407. When data ready signal 408 is asserted, the 
2J) data on bus 403 is latched into data output circuit 401, and 
control circuit 405 enters state 452 in which the framing 
sequence is transmitted. In this embodiment, if the last data 
□ packet was sent more than two transmit clock cycles ago, only a 
logic "1" bit is transmitted in the next two cycles. Otherwise, 
25 a logic "0" is transmitted for two transmit clock cycles to 
ensure that the packets are separated by at least two clock 
cycles. After the framing sequence is transmitted, control 
circuit 405 enters state 453 in which the data latched into data 
output circuit 401 is serialized and transmitted on serial line 
30 407 bit by bit, each bit being sent over two transmit clock 
cycles. At the end of data transmission, the parity data 
computed in parity generation circuit 402 is transmitted on 
serial line 407. The data packet is at that point completely 
transmitted. Control circuit 405 then returns to idle state 
35 451. A reset signal can be provided to reset control circuit 
405 back to state 451 at any time. 
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Figure 5a is a block diagram of receiver circuit 500 in 
accordance with one embodiment of the present invention. As 
shown in Figure 5a, serial data 507 is sampled by serially 
connected flip-flops 501 and 502 at the falling edges of clock 
5 signal 509, which has the same frequency as the transmit clock 
signal of transmitter 400 discussed above. The sampled signal 
(at terminal 512) is provided to phase detector 503 for 
detecting the framing sequence of a data packet. Data receiving 
circuit 504 and parity detection circuit 505 sample serial data 
10 507 at half the clock rate of clock signal 510 upon detection of 
the framing sequence by phase detector 503. In one embodiment, 
clock signal 510 is a complementary signal of clock signal 509. 
In that embodiment, data receiving circuit 504 begins to sample 
j =f serial data 507 at every second clock edge of clock signal 510, 
If! after phase detector 503 detects the first logic N> 1" at terminal 
LJ 512. If parity detection circuit 511 does not detect an error 

in serial data 507, data receiving circuit 508 provides a 
:;ri parallel output on data bus 507. Control circuit 506 controls 

the operations of phase detector circuit 503, data receiving 
2$> circuit 504 and parity detection circuit 505. 

UJ Figure 5b shows state diagram 550 that illustrates the 

control operations of control circuit 506. Initially, control 

q circuit 505 waits in state 551 for a "go" or ready signal to be 
asserted. When the go signal is asserted, control circuit 505 

25 enters state 552 in which phase detector circuit 503 samples 

terminal 512 to detect the framing sequence. Once the framing 
sequence is detected, control circuit 505 enters state 553 in 
which data receiving circuit 504 and parity detection circuit 
505 samples serial data 507 until the expected number of bits in 

3 0 the data packet are sampled. Control circuit 505 then returns 
to state 551 for at least two cycles until the go signal is 
asserted. A reset signal can be provided to reset control 
circuit 506 back to state 551 at any time. 

Transmitter circuit 400 and receiver circuit 500 can be 
35 incorporated in an emulation circuit where data signals are to 
be sent between circuit boards that may reside in different 
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chassis of the emulation system. Figure 6 shows circuit 600 
that can be configured in an emulation circuit consisting of 
multiple circuit boards to effectuate data transfer. As shown 
in Figure 6, circuit 600 includes portions 601 and 602 that are 
5 to be configured in circuit boards of different chassis. Data 
is transmitted serially from portion 601 to portion 602 through 
connecting wire 603, using the protocol described above. 
Portion 601 includes a number of input buffers labeled 604i to 
604k, corresponding to logic signals to be distribution to other 
10 parts of the emulation circuit according to their relevance for 
system clock periods ("epochs") i to k. Typically, the logic 
circuit signals in buffers 604i to 604k are collected from the 
user circuit to be emulated. During emulation, data signals 
organized by their respective epochs appear on corresponding 
A% connecting terminals 608i to 608j at each clock period of the 
LJ virtual clock. Some of the signals at terminals 608i to 608j 
" z t are fed back into circuits in portion 601 via IO blocks 605i to 
ilfi 605j . The signals at terminals 608i to 608j are also made 
j " y available for transmission to portion 602 of the emulation 
kp circuit using transmitters 606i to 606j . Transmitters 606i to 
UJ 606k can each be implemented by transmitter 400 described above . 
j=S The output values of transmitters 606i to 606j are transmitted 
□ to portion 602 of emulation circuit 600 according to the 

transmit clock over connecting wire603. Multiplexor 607 selects 
25 the output data of transmitters 606i to 606 j onto connecting 

wire 603. In this embodiment, the transmit clock transmits at 
one half the frequency of the virtual clock. However, a phase- 
locked loop can be used create a clock signal which is double 
the frequency of the virtual clock. Such a clock signal would 
30 allow transmission to take place at the virtual clock rate. 

In portion 603 of emulation circuit 600, data received on 
connecting wire 603 is demultiplexed according to epoch and 
provided to receivers 611i to 611k respectively. Receivers 611i 
to 611k can each be implemented by receiver 500 described above. 
35 The output values of receivers 611i to 611k are provided to user 
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logic circuit 612 along with corresponding signals in IO blocks 
610i to 610k. 

Although the present invention is illustrated above using 
examples of wires carrying data in one direction, the present 
5 invention allows data to be communicated in both directions 
using one or more wires, by providing both transmitters and 
receivers at each interface. 

Figure 7 shows system 700 including emulation system 701, 

controller 702, and host system 750, in another embodiment of 
10 the present invention. As shown in Figure 7, emulation system 

701 and controller 702 communicates over a bidirectional serial 

interface 730. An arbitration procedure between control 
: L 3 circuits 714 and 724 of emulation system 701 and controller 702, 

respectively, determines the direction of data flow between 
1=5 emulation system 701 and controller 702. Control circuits 714 

and 724 control their respective transmitter and receiver to 
j s 'j effectuate the data transfer. Controller 702 and emulation 

system 701 are sufficiently separated from each other to be 
w effectively asynchronous to each other. Thus, the protocol of 
2b the present invention described above for communication between 

substantially asynchronous systems is applicable to 
jl= communication on serial interface 730. Host system 750 

communicates with controller 7 02 over an industry standard bus 

interface 751, such as the PCI bus. 

25 Emulation system 701 includes user logic circuits 712, 

input/output buffers 713-1 to 713-i, transmitter 710, receiver 
711 and control circuit 714. During operation, data to be 
transmitted from emulation system 701 to controller 702 or host 
system 750 are provided over input/output buffers 713-1 to 713-i 

30 to be transmitted over serial interface 730 to controller 702 
and host system 750. Data from controller 702 or host system 
750 are provided over serial interface 73 0 to receiver 711, 
which then provides the data to user logic circuits 712. User 
logic circuits 712, input/output buffers 713-1 to 713-i, 

35 transmitter 710, receiver 711 and control circuit 714 can all be 
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configured in the programmable logic circuits (e.g., FPGAs) of 
emulation system 701. 

As shown in Figure 7, in controller 702 , first- in- first-out 
(FIFO) memories are provided to allow data communicated between 
host system 750 and controller 702 over bus interface 751 to be 
queued at controller 702. 

The detailed description above is provided to illustrate 
specific embodiments of the present invention and is not 
intended to be limiting. Numerous modifications and variations 
within the scope of the present invention are possible. The 
present invention is set forth in the following claims. 
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